De-biasing the Lasso: Optimal Sample Size for Gaussian Designs

Performing statistical inference in high-dimensional models is an outstanding challenge. A major source of difficulty is the absence of precise information on the distribution of high-dimensional regularized estimators. Here, we consider linear regression in the high-dimensional regime p n and the Lasso estimator. In this context, we would like to perform inference on a high-dimensional parameters vector θ∗ ∈ R. Important progress has been achieved in computing confidence intervals and p-values for single coordinates θ∗ i , i ∈ {1, . . . , p}. A key role in these new inferential methods is played by a certain de-biased (or de-sparsified) estimator θ̂ that is constructed from the Lasso estimator. Earlier work establishes that, under suitable assumptions on the design matrix, the coordinates of θ̂ are asymptotically Gaussian provided the true parameters vector θ∗ is s0-sparse with s0 = o( √ n/ log p). The condition s0 = o( √ n/ log p) is considerably stronger than the one required for consistent estimation, namely s0 = o(n/ log p). Here we consider Gaussian designs with known or unknown population covariance. When the covariance is known, we prove that the de-biased estimator is asymptotically Gaussian under the nearly optimal condition s0 = o(n/(log p) ). Note that earlier work was limited to s0 = o( √ n/ log p) even for perfectly known covariance. The same conclusion holds if the population covariance is unknown but can be estimated sufficiently well, e.g. because its inverse is very sparse. For intermediate regimes, we describe the trade-off between sparsity in the coefficients θ∗, and sparsity in the inverse covariance of the design. ∗Data Sciences and Operations Department, Marshall School of Business, University of Southern California, Email: [email protected] †Department of Electrical Engineering and Department of Statistics, Stanford University. Email: montanar@ stanford.edu